Constrained object correction for a segmented image

ABSTRACT

Disclosed is a computer-implemented method of segmenting a medical patient image using an atlas and relating the segmentation result to a model of possible geometric changes to the segmentation result (e.g. for correcting the position of the segmentation of anatomical structures) which consider for example anatomical limitations. The thus-related segmentation result may be used as a basis for changing and/or correcting the position, shape and/or orientation of at least parts of the segmentation result, e.g. by user interaction. The invention also relates to an atlas data set comprising information such as values of the variables of the model of possible geometric changes in relation to the positions of anatomical structures in the atlas.

FIELD OF THE INVENTION

The present invention relates to a computer-implemented method fordetermining the geometric transformability of a segmentation of amedical patient image, a corresponding computer program, anon-transitory program storage medium storing such a program and acomputer for executing the program, as well as a medical systemcomprising an electronic data storage device and the aforementionedcomputer.

TECHNICAL BACKGROUND

The result of automatically segmenting a representation of an anatomicalstructure in an image often is not perfect and modification by a usermay be required. Current modification tools normally modify only oneobject. This is done by simply brushing, by brushing constrained by theedges of the gray value images, by more intelligent brushing (smartbrush) or by deforming (smart shaper). Known methods generally modifyonly one object at a time. Anatomic constraints with respect to otherobjects are thereby not considered. For example, if one modifies theprostate, the bladder and the rectum must also be moved accordingly.Such a consistent modification of multiple objects is not possible withthe existing tools.

The present invention has the object of providing a method whichsupports simultaneous modification of several structures, but respectsanatomical constraints

The present invention can be used for electronic data processing relatedimage-based pre-planning which may be done e.g. using the Elementsmodules, a product of Brainlab AG, e.g. in connection with a system forimage-guided radiotherapy such as VERO® and ExacTrac®, both alsoproducts of Brainlab AG.

Aspects of the present invention, examples and exemplary steps and theirembodiments are disclosed in the following. Different exemplary featuresof the invention can be combined in accordance with the inventionwherever technically expedient and feasible.

EXEMPLARY SHORT DESCRIPTION OF THE INVENTION

In the following, a short description of the specific features of thepresent invention is given which shall not be understood to limit theinvention only to the features or a combination of the featuresdescribed in this section.

The disclosed method encompasses segmenting a medical patient imageusing an atlas and relating the segmentation result to a model ofpossible geometric changes to the segmentation result (e.g. forcorrecting the position of the segmentation of anatomical structures)which consider for example anatomical limitations. The thus-relatedsegmentation result may be used as a basis for changing and/orcorrecting the position, shape and/or orientation of at least parts ofthe segmentation result, e.g. by user interaction. The invention alsorelates to an atlas data set comprising information such as values ofthe variables of the model of possible geometric changes in relation tothe positions of anatomical structures in the atlas.

GENERAL DESCRIPTION OF THE INVENTION

In this section, a description of the general features of the presentinvention is given for example by referring to possible embodiments ofthe invention.

In general, the invention reaches the aforementioned object byproviding, in a first aspect, a computer-implemented medical method fordetermining the geometric transformability of a segmentation of amedical patient image. The method comprises executing, on at least oneprocessor of at least one computer (for example at least one computerbeing part of the navigation system or planning system), the followingexemplary steps which are executed by the at least one processor.

In a (for example first) exemplary step, patient image data is acquiredwhich describes a patient image of an anatomical body part of a patient.The patient image data has for example been generated beforehand (i.e.before execution of the method according to the first aspect starts) oras a part of the disclosed method according to the first aspect. Thepatient image data has been or is generated by applying a medicalimaging modality to the anatomical body part. For example, the patientimage data is two-dimensional medical image data such as a radiographyor three-dimensional image data such as a magnetic resonance tomographyor a computed x-ray tomography or a sonography. The anatomical body partmay essentially be any part of a patient body and may include at leastone of bony (including cartilage) or soft tissue. In examples, theanatomical body part comprises a plurality of (i.e. at least two)anatomical structures which may be physically connected to each other(such as the prostate and the rectum).

In a (for example second) exemplary step, atlas data is acquired whichdescribes an image-based model of the anatomical body part and at leastone value of at least one parameter (which in the framework of thisdisclosure may also be called variable) of a geometric transformabilitymodel of the anatomical body part. The at least one value of the atleast one parameter may be used as an input quantity to thecorresponding parameter used in the geometric transformability model.The at least one parameter may represent a physical variable for exampleas a numeric variable (e.g. of integer or real type, i.e. a numericvalue of the physical variable) but may alternatively or additionally beof a different data type such as a logic variable (e.g. of Boole type),e.g. for executing a logical operation when conducting a computation onthe geometric transformability model.

In a (for example third) exemplary step, transformability model data isacquired which describes the geometric transformability model of theanatomical body part (for example, of a digital image representation ofthe anatomical body part). The geometric transformability modeldescribes a geometric transformability of the anatomical body partand/or of the image representation of the anatomical body part.Specifically, it describes the geometric transformability of each of theanatomical structures which are part of the anatomical body part. Theterm of geometric transformability encompasses for example the degreesof freedom (for example, at least one of possible translations orrotations) of movement of each anatomical structure and for exampleadditionally or alternatively at least one of an elasticity orcompressibility of each anatomical structure (defined e.g. by a springmodel or finite element model). The term of geometric transformationthus encompasses movement (i.e. at least one of translation androtation, for example rigid translation or rotation) of each anatomicalstructure and for example additionally or alternatively at least one ofan elastic deformation or compression of the anatomical body part or atleast an anatomical structure included in the anatomical body part).Alternatively or additionally, the geometric transformabilityencompasses a definition of whether the anatomical structure can bepenetrated by another anatomical structure (for example, if theanatomical structure comprises bony tissue, the geometrictransformability model may define that the bony tissue cannot bepenetrated by soft tissue, i.e. that a soft tissue anatomical structurecannot be at the same position as the bony tissue anatomical structure).The transformability model data may be part of the atlas data (e.g.stored in the same data set) or may form a separate data set which forexample is linked (e.g. concerning positional information) to theimage-based model of the anatomical body part described by the atlasdata.

For example, the geometric transformability model describes a movementinteraction (for example, a possible movement interaction, e.g.simultaneous movement, an unchanged motion state, a restricted movement,a sliding movement) between the relative positions of at leastsubstantially disjunct parts of the image-based model of the anatomicalbody part which for example resemble the anatomical structures which arepart of the anatomical body part.

For example, the geometric transformability model describes a movability(for example at least one of translational or rotational degrees offreedom, of the anatomical body part i.e. the possibility of moving ofthe anatomical body part, specifically individually for each of theanatomical structures) and alternatively or additionally an elastic orinelastic deformation model of the image-based model of the anatomicalbody part, for example a distance-dependent deformation function of theimage-based model of the anatomical body part (e.g. of the anatomicalstructures defined in the image-based model).

For example, the geometric transformability model is defined on a grid,for example numeric grid, which is for example linked to at leastsubstantially disjunct parts of the anatomical body part. The griddefines nodes which for example define positions at which results ofpossible geometric transformation are calculated on the basis of e.g.computing the physics of the geometric transformability model.

For example, the geometric transformability model comprises a shapemodel describing allowed shapes and movement directions of at leastsubstantially disjunct parts of the image-based model of the anatomicalbody part.

For example, boundary conditions of the geometric transformability modelare defined within and/or between at least substantially disjunct partsof the anatomical body part. For example, the geometric transformabilitymodel contains boundary conditions defining for example anatomicalrealities such as a physical connection between anatomical structureswhich means that if geometry (including for example the position) of oneof the structures is transformed, the geometry (including for examplethe position) of another anatomical structure connected to thatanatomical structure has to be correspondingly transformed.Alternatively or additionally, the boundary conditions may define thatanatomical structures may not intersect and/or penetrate one another(i.e. may not have the same position) as a result of transforming thegeometry of at least one of the anatomical structures, or that certainanatomical structure may only slide along another or always have to befixed relative to a global coordinate system. In one exemplaryimplementation, a user may violate the boundary conditions set by themethod, e.g. in order to manually correct the segmentation result. Apossible case would be that the segmentation result defines an artery tolie outside of the spine even though it should be located inside thespine. Even though the spine would be defined as bony tissue whichcannot be penetrated by soft tissue such as the artery, the user wouldbe allowed to for example drag and drop the segmented imagerepresentation of the artery into the segmented image representation ofthe spine. For example, the boundary condition prohibiting such anaction and automatically set by the method can be overridden if it hasbeen deactivated, e.g. by user input.

In a (for example fourth) exemplary step, segmented image data isdetermined, based on the patient image data and the atlas data and thetransformability model data, which describes a segmentation of thepatient image and a geometric relation between the geometrictransformability model and the segmentation. For example, the segmentedimage data is determined by determining a positional transformationbetween the patient image of the anatomical body part and theimage-based model of the anatomical body part. For example, thepositional transformation (which may take the form of a linear transformwhich can be defined by a mapping matrix) is determined by applying animage matching algorithm, for example an elastic or rigid image fusionalgorithm, to the patient image data and the atlas data. The positionaltransformation thus represents an image registration between the patientimage described by the patient image data and the image-based modeldescribed by the atlas. The geometric relation between the geometrictransformability model may be established by pre-defining a relationshipbetween anatomical structures described the atlas data and correspondingportions of the geometric transformability model. For example, ageometric (e.g. positional) relation between the geometrictransformability model and the anatomical structures segmented in thepatient image is established based on (e.g. via) the positionaltransformation between the patient image and the image-based model ofthe atlas data.

In a (for example fifth) exemplary step, assigned transformability modeldata is determined, based on the segmented image data and the atlasdata, which describes an assignment of the at least one value of the atleast one parameter to the geometric relation between the geometrictransformability model and the segmentation. For example, the assignmentof the at least one value of the at least one parameter to the geometricrelation between the geometric transformability model and thesegmentation is determined by inputting the at least one value of the atleast one parameter to the transformability model. The assignmenttherefore constitutes an application of the geometric transformabilitymodel to the segmented patient image. For example, the assignment isimplemented by relating the positions of the nodes of the geometrictransformability model to the positions of anatomical structuresrepresented by the segmented patient image (such as a meshing, e.g. agrid generation leading to a grid representation of the anatomical bodypart, and mapping of at least one physical quantity and/or mechanicalboundary conditions to at least one predetermined part of the meshand/or grid).

In a second aspect, the invention is directed to a computer-implementedmedical data processing method of adapting a segmentation of a medicalpatient image, the method comprising the following steps:

-   -   a) executing the method according to the first aspect;    -   b) determining, based on the segmented image data and the        assigned transformability model data, changed (e.g. manually or        automatically changed) segmented image data describing a changed        segmentation of the patient image.

This allows a user to for example manually change and/or correct theresult of the segmentation of the patient image while automaticallyrespecting the boundary conditions of the geometric transformabilitymodel. For example, the user may use a pointing tool such as a mousecursor or a touch screen for selecting a part of the assigned geometrictransformability model (e.g. a node of the above-described grid) forchanging its position as desired. Alternatively or additionally, thechange and/or correction of the segmented patient image may bedetermined automatically. The distorted grid (i.e. the changedsegmentation and the resulting change of the assigned geometrictransformability model) can also be used to distort the registration(the atlas-patient mapping, i.e. the transformation between the patientimage described by the patient image data and the image-based modeldescribed by the atlas data), which was the basis for the segmentation:Reg_(new)=Distortion* Reg_(old) (where Reg_(new) is the newatlas-patient mapping after correcting the segmentation, Distortion isthe change to the segmentation and/or the geometric transformabilitymodel, and Reg_(old) is the atlas-patient mapping before correcting thesegmentation).

In a third aspect, the invention is directed to a computer programwhich, when running on at least one processor (for example, a processor)of at least one computer (for example, a computer) or when loaded intoat least one memory (for example, a memory) of at least one computer(for example, a computer), causes the at least one computer to performthe above-described method according to the first or second aspect. Theinvention may alternatively or additionally relate to a (physical, forexample electrical, for example technically generated) signal wave, forexample a digital signal wave, carrying information which represents theprogram, for example the aforementioned program, which for examplecomprises code means which are adapted to perform any or all of thesteps of the method according to the first or second aspect. A computerprogram stored on a disc is a data file, and when the file is read outand transmitted it becomes a data stream for example in the form of a(physical, for example electrical, for example technically generated)signal. The signal can be implemented as the signal wave which isdescribed herein. For example, the signal, for example the signal waveis constituted to be transmitted via a computer network, for exampleLAN, WLAN, WAN, mobile network, for example the internet. For example,the signal, for example the signal wave, is constituted to betransmitted by optic or acoustic data transmission. The inventionaccording to the third aspect therefore may alternatively oradditionally relate to a data stream representative of theaforementioned program.

In a fourth aspect, the invention is directed to a non-transitorycomputer-readable program storage medium on which the program accordingto the third aspect is stored.

In a fifth aspect, the invention is directed to at least one computer(for example, a computer), comprising at least one processor (forexample, a processor) and at least one memory (for example, a memory),wherein the program according to the third aspect is running on theprocessor or is loaded into the memory, or wherein the at least onecomputer comprises the computer-readable program storage mediumaccording to the fourth aspect.

In a sixth aspect, the invention is directed to a system for planning amedical procedure, comprising:

-   -   a) the at least one computer according to the fifth aspect;    -   b) at least one electronic data storage device storing at least        the patient image data, the atlas data and the transformability        model data; and    -   c) a display device for displaying at least one of the patient        image data, the segmented image data or, as far as the program        run by the computer or loaded into the memory of the computer        causes the computer to perform the method according to the        second aspect, the changed segmented image data;    -   d) wherein the at least one computer is operably coupled        -   to the at least one electronic data storage device for            acquiring, from the at least one data storage device, at            least the patient image data, the atlas data and the            transformability model data, and        -   to the display device for issuing, to the display device, a            command for displaying at least one of the patient image            data, the segmented image data or, as far as the program run            by the computer or loaded into the memory of the computer            causes the computer to perform the method of claim 10, the            changed segmented image data.

In a seventh aspect, the invention is directed to a data set (forexample, an electronic and/or digital data set) comprising atlas data,the atlas data describing an image-based model of the anatomical bodypart, and the data set comprising at least one of

-   -   the transformability model data which describes a geometric        transformability model of an anatomical body part; or    -   information about at least one value of the at least one        parameter of the geometric transformability model for        geometrically transforming the anatomical body part (for        example, a digital image representation of the anatomical body        part).

In an example of the data set according to the seventh aspect, thegeometric transformability model describes a movability, for example atleast one of translational or rotational degrees of freedom, of theanatomical body part or an elastic or inelastic deformation model of theimage-based model of the anatomical body part, for example adistance-dependent deformation function of the image-based model of theanatomical body part.

In an eighth aspect, the invention is directed to a non-transitory, forexample digital or electronic, data storage medium on which the data setaccording to the seventh aspect is stored, and/or a digital data filecomprising the data set according to the seventh aspect, and/or at leastone of a data sequence or signal wave or data stream, for exampledigital or electronic signal wave or digital or electronic data stream,representing the data set according to the seventh aspect.

In a ninth aspect, the present invention is directed to the use of thesystem according to the sixth aspect for planning a medical procedure.The use comprises for example at least causing the computer of thesystem to execute the method according to the first or second aspect orthe program according to the third aspect.

For example, the invention does not involve or in particular comprise orencompass an invasive step which would represent a substantial physicalinterference with the body requiring professional medical expertise tobe carried out and entailing a substantial health risk even when carriedout with the required professional care and expertise.

DEFINITIONS

In this section, definitions for specific terminology used in thisdisclosure are offered which also form part of the present disclosure.

Computer Implemented Method

The method in accordance with the invention is for example a computerimplemented method. For example, all the steps or merely some of thesteps (i.e. less than the total number of steps) of the method inaccordance with the invention can be executed by a computer (forexample, at least one computer). An embodiment of the computerimplemented method is a use of the computer for performing a dataprocessing method. An embodiment of the computer implemented method is amethod concerning the operation of the computer such that the computeris operated to perform one, more or all steps of the method.

The computer for example comprises at least one processor and forexample at least one memory in order to (technically) process the data,for example electronically and/or optically. The processor being forexample made of a substance or composition which is a semiconductor, forexample at least partly n- and/or p-doped semiconductor, for example atleast one of II-, III-, IV-, V-, VI-semiconductor material, for example(doped) silicon and/or gallium arsenide. The calculating or determiningsteps described are for example performed by a computer. Determiningsteps or calculating steps are for example steps of determining datawithin the framework of the technical method, for example within theframework of a program. A computer is for example any kind of dataprocessing device, for example electronic data processing device. Acomputer can be a device which is generally thought of as such, forexample desktop PCs, notebooks, netbooks, etc., but can also be anyprogrammable apparatus, such as for example a mobile phone or anembedded processor. A computer can for example comprise a system(network) of “sub-computers”, wherein each sub-computer represents acomputer in its own right. The term “computer” includes a cloudcomputer, for example a cloud server. The term “cloud computer” includesa cloud computer system which for example comprises a system of at leastone cloud computer and for example a plurality of operativelyinterconnected cloud computers such as a server farm. Such a cloudcomputer is preferably connected to a wide area network such as theworld wide web (WWW) and located in a so-called cloud of computers whichare all connected to the world wide web. Such an infrastructure is usedfor “cloud computing”, which describes computation, software, dataaccess and storage services which do not require the end user to knowthe physical location and/or configuration of the computer delivering aspecific service. For example, the term “cloud” is used in this respectas a metaphor for the Internet (world wide web). For example, the cloudprovides computing infrastructure as a service (IaaS). The cloudcomputer can function as a virtual host for an operating system and/ordata processing application which is used to execute the method of theinvention. The cloud computer is for example an elastic compute cloud(EC2) as provided by Amazon Web Services™. A computer for examplecomprises interfaces in order to receive or output data and/or performan analogue-to-digital conversion. The data are for example data whichrepresent physical properties and/or which are generated from technicalsignals. The technical signals are for example generated by means of(technical) detection devices (such as for example devices for detectingmarker devices) and/or (technical) analytical devices (such as forexample devices for performing (medical) imaging methods), wherein thetechnical signals are for example electrical or optical signals. Thetechnical signals for example represent the data received or outputtedby the computer. The computer is preferably operatively coupled to adisplay device which allows information outputted by the computer to bedisplayed, for example to a user. One example of a display device is avirtual reality device or an augmented reality device (also referred toas virtual reality glasses or augmented reality glasses) which can beused as “goggles” for navigating. A specific example of such augmentedreality glasses is Google Glass (a trademark of Google, Inc.). Anaugmented reality device or a virtual reality device can be used both toinput information into the computer by user interaction and to displayinformation outputted by the computer. Another example of a displaydevice would be a standard computer monitor comprising for example aliquid crystal display operatively coupled to the computer for receivingdisplay control data from the computer for generating signals used todisplay image information content on the display device. A specificembodiment of such a computer monitor is a digital lightbox. An exampleof such a digital lightbox is Buzz®, a product of Brainlab AG. Themonitor may also be the monitor of a portable, for example handheld,device such as a smart phone or personal digital assistant or digitalmedia player.

The invention also relates to a program which, when running on acomputer, causes the computer to perform one or more or all of themethod steps described herein and/or to a program storage medium onwhich the program is stored (in particular in a non-transitory form)and/or to a computer comprising said program storage medium and/or to a(physical, for example electrical, for example technically generated)signal wave, for example a digital signal wave, carrying informationwhich represents the program, for example the aforementioned program,which for example comprises code means which are adapted to perform anyor all of the method steps described herein.

Within the framework of the invention, computer program elements can beembodied by hardware and/or software (this includes firmware, residentsoftware, micro-code, etc.). Within the framework of the invention,computer program elements can take the form of a computer programproduct which can be embodied by a computer-usable, for examplecomputer-readable data storage medium comprising computer-usable, forexample computer-readable program instructions, “code” or a “computerprogram” embodied in said data storage medium for use on or inconnection with the instruction-executing system. Such a system can be acomputer; a computer can be a data processing device comprising meansfor executing the computer program elements and/or the program inaccordance with the invention, for example a data processing devicecomprising a digital processor (central processing unit or CPU) whichexecutes the computer program elements, and optionally a volatile memory(for example a random access memory or RAM) for storing data used forand/or produced by executing the computer program elements. Within theframework of the present invention, a computer-usable, for examplecomputer-readable data storage medium can be any data storage mediumwhich can include, store, communicate, propagate or transport theprogram for use on or in connection with the instruction-executingsystem, apparatus or device. The computer-usable, for examplecomputer-readable data storage medium can for example be, but is notlimited to, an electronic, magnetic, optical, electromagnetic, infraredor semiconductor system, apparatus or device or a medium of propagationsuch as for example the Internet. The computer-usable orcomputer-readable data storage medium could even for example be paper oranother suitable medium onto which the program is printed, since theprogram could be electronically captured, for example by opticallyscanning the paper or other suitable medium, and then compiled,interpreted or otherwise processed in a suitable manner. The datastorage medium is preferably a non-volatile data storage medium. Thecomputer program product and any software and/or hardware described hereform the various means for performing the functions of the invention inthe example embodiments. The computer and/or data processing device canfor example include a guidance information device which includes meansfor outputting guidance information. The guidance information can beoutputted, for example to a user, visually by a visual indicating means(for example, a monitor and/or a lamp) and/or acoustically by anacoustic indicating means (for example, a loudspeaker and/or a digitalspeech output device) and/or tactilely by a tactile indicating means(for example, a vibrating element or a vibration element incorporatedinto an instrument). For the purpose of this document, a computer is atechnical computer which for example comprises technical, for exampletangible components, for example mechanical and/or electroniccomponents. Any device mentioned as such in this document is a technicaland for example tangible device.

Acquiring Data

The expression “acquiring data” for example encompasses (within theframework of a computer implemented method) the scenario in which thedata are determined by the computer implemented method or program.Determining data for example encompasses measuring physical quantitiesand transforming the measured values into data, for example digitaldata, and/or computing (and e.g. outputting) the data by means of acomputer and for example within the framework of the method inaccordance with the invention. A step of “determining” a describedherein for example comprises or consists of issuing a command to performthe determination described herein. For example, the step comprises orconsists of issuing a command to cause computer, for example a remotecomputer, for example a remote server, for example in the cloud, toperform the determination. Alternatively or additionally, a step of“determining” as described herein for example comprises or consists ofreceiving the data resulting from the determination described herein.The meaning of “acquiring data” also for example encompasses thescenario in which the data are received or retrieved by (e.g. input to)the computer implemented method or program, for example from anotherprogram, a previous method step or a data storage medium, for examplefor further processing by the computer implemented method or program.Generation of the data to be acquired may but need not be part of themethod in accordance with the invention. The expression “acquiring data”can therefore also for example mean waiting to receive data and/orreceiving the data. The received data can for example be inputted via aninterface. The expression “acquiring data” can also mean that thecomputer implemented method or program performs steps in order to(actively) receive or retrieve the data from a data source, for instancea data storage medium (such as for example a ROM, RAM, database, harddrive, etc.), or via the interface (for instance, from another computeror a network). The data acquired by the disclosed method or device,respectively, may be acquired from a database located in a data storagedevice which is operably to a computer for data transfer between thedatabase and the computer, for example from the database to thecomputer. The computer acquires the data for use as an input for stepsof determining data. The determined data can be output again to the sameor another database to be stored for later use. The database or databaseused for implementing the disclosed method can be located on networkdata storage device or a network server (for example, a cloud datastorage device or a cloud server) or a local data storage device (suchas a mass storage device operably connected to at least one computerexecuting the disclosed method). The data can be made “ready for use” byperforming an additional step before the acquiring step. In accordancewith this additional step, the data are generated in order to beacquired. The data are for example detected or captured (for example byan analytical device). Alternatively or additionally, the data areinputted in accordance with the additional step, for instance viainterfaces. The data generated can for example be inputted (for instanceinto the computer). In accordance with the additional step (whichprecedes the acquiring step), the data can also be provided byperforming the additional step of storing the data in a data storagemedium (such as for example a ROM, RAM, CD and/or hard drive), such thatthey are ready for use within the framework of the method or program inaccordance with the invention. The step of “acquiring data” cantherefore also involve commanding a device to obtain and/or provide thedata to be acquired. In particular, the acquiring step does not involvean invasive step which would represent a substantial physicalinterference with the body, requiring professional medical expertise tobe carried out and entailing a substantial health risk even when carriedout with the required professional care and expertise. In particular,the step of acquiring data, for example determining data, does notinvolve a surgical step and in particular does not involve a step oftreating a human or animal body using surgery or therapy. In order todistinguish the different data used by the present method, the data aredenoted (i.e. referred to) as “XY data” and the like and are defined interms of the information which they describe, which is then preferablyreferred to as “XY information” and the like.

Image Registration

Image registration is the process of transforming different sets of datainto one coordinate system. The data can be multiple photographs and/ordata from different sensors, different times or different viewpoints. Itis used in computer vision, medical imaging and in compiling andanalysing images and data from satellites. Registration is necessary inorder to be able to compare or integrate the data obtained from thesedifferent measurements.

Atlas/Atlas Segmentation

Preferably, atlas data is acquired which describes (for example defines,more particularly represents and/or is) a general three-dimensionalshape of the anatomical body part. The atlas data therefore representsan atlas of the anatomical body part. An atlas typically consists of aplurality of generic models of objects, wherein the generic models ofthe objects together form a complex structure. For example, the atlasconstitutes a statistical model of a patient's body (for example, a partof the body) which has been generated from anatomic information gatheredfrom a plurality of human bodies, for example from medical image datacontaining images of such human bodies. In principle, the atlas datatherefore represents the result of a statistical analysis of suchmedical image data for a plurality of human bodies. This result can beoutput as an image—the atlas data therefore contains or is comparable tomedical image data. Such a comparison can be carried out for example byapplying an image fusion algorithm which conducts an image fusionbetween the atlas data and the medical image data. The result of thecomparison can be a measure of similarity between the atlas data and themedical image data. The atlas data comprises image information (forexample, positional image information) which can be matched (for exampleby applying an elastic or rigid image fusion algorithm) for example toimage information (for example, positional image information) containedin medical image data so as to for example compare the atlas data to themedical image data in order to determine the position of anatomicalstructures in the medical image data which correspond to anatomicalstructures defined by the atlas data.

The human bodies, the anatomy of which serves as an input for generatingthe atlas data, advantageously share a common feature such as at leastone of gender, age, ethnicity, body measurements (e.g. size and/or mass)and pathologic state. The anatomic information describes for example theanatomy of the human bodies and is extracted for example from medicalimage information about the human bodies. The atlas of a femur, forexample, can comprise the head, the neck, the body, the greatertrochanter, the lesser trochanter and the lower extremity as objectswhich together make up the complete structure. The atlas of a brain, forexample, can comprise the telencephalon, the cerebellum, thediencephalon, the pons, the mesencephalon and the medulla as the objectswhich together make up the complex structure. One application of such anatlas is in the segmentation of medical images, in which the atlas ismatched to medical image data, and the image data are compared with thematched atlas in order to assign a point (a pixel or voxel) of the imagedata to an object of the matched atlas, thereby segmenting the imagedata into objects.

For example, the atlas data includes information of the anatomical bodypart. This information is for example at least one of patient-specific,non-patient-specific, indication-specific or non-indication-specific.The atlas data therefore describes for example at least one of apatient-specific, non-patient-specific, indication-specific ornon-indication-specific atlas. For example, the atlas data includesmovement information indicating a degree of freedom of movement of theanatomical body part with respect to a given reference (e.g. anotheranatomical body part). For example, the atlas is a multimodal atlaswhich defines atlas information for a plurality of (i.e. at least two)imaging modalities and contains a mapping between the atlas informationin different imaging modalities (for example, a mapping between all ofthe modalities) so that the atlas can be used for transforming medicalimage information from its image depiction in a first imaging modalityinto its image depiction in a second imaging modality which is differentfrom the first imaging modality or to compare (for example, match orregister) images of different imaging modality with one another.

Imaging Methods

In the field of medicine, imaging methods (also called imagingmodalities and/or medical imaging modalities) are used to generate imagedata (for example, two-dimensional or three-dimensional image data) ofanatomical structures (such as soft tissues, bones, organs, etc.) of thehuman body. The term “medical imaging methods” is understood to mean(advantageously apparatus-based) imaging methods (for example so-calledmedical imaging modalities and/or radiological imaging methods) such asfor instance computed tomography (CT) and cone beam computed tomography(CBCT, such as volumetric CBCT), x-ray tomography, magnetic resonancetomography (MRT or MRI), conventional x-ray, sonography and/orultrasound examinations, and positron emission tomography. For example,the medical imaging methods are performed by the analytical devices.Examples for medical imaging modalities applied by medical imagingmethods are: X-ray radiography, magnetic resonance imaging, medicalultrasonography or ultrasound, endoscopy, elastography, tactile imaging,thermography, medical photography and nuclear medicine functionalimaging techniques as positron emission tomography (PET) andSingle-photon emission computed tomography (SPECT), as mentioned byWikipedia. The image data thus generated is also termed “medical imagingdata”. Analytical devices for example are used to generate the imagedata in apparatus-based imaging methods. The imaging methods are forexample used for medical diagnostics, to analyse the anatomical body inorder to generate images which are described by the image data. Theimaging methods are also for example used to detect pathological changesin the human body. However, some of the changes in the anatomicalstructure, such as the pathological changes in the structures (tissue),may not be detectable and for example may not be visible in the imagesgenerated by the imaging methods. A tumour represents an example of achange in an anatomical structure. If the tumour grows, it may then besaid to represent an expanded anatomical structure. This expandedanatomical structure may not be detectable; for example, only a part ofthe expanded anatomical structure may be detectable. Primary/high-gradebrain tumours are for example usually visible on MRI scans when contrastagents are used to infiltrate the tumour. MRI scans represent an exampleof an imaging method. In the case of MRI scans of such brain tumours,the signal enhancement in the MRI images (due to the contrast agentsinfiltrating the tumour) is considered to represent the solid tumourmass. Thus, the tumour is detectable and for example discernible in theimage generated by the imaging method. In addition to these tumours,referred to as “enhancing” tumours, it is thought that approximately 10%of brain tumours are not discernible on a scan and are for example notvisible to a user looking at the images generated by the imaging method.

Mapping

Mapping describes a transformation (for example, linear transformation)of an element (for example, a pixel or voxel), for example the positionof an element, of a first data set in a first coordinate system to anelement (for example, a pixel or voxel), for example the position of anelement, of a second data set in a second coordinate system (which mayhave a basis which is different from the basis of the first coordinatesystem). In one embodiment, the mapping is determined by comparing (forexample, matching) the color values (for example grey values) of therespective elements by means of an elastic or rigid fusion algorithm.The mapping is embodied for example by a transformation matrix (such asa matrix defining an affine transformation).

Elastic Fusion, Image Fusion/Morphing, Rigid

Image fusion can be elastic image fusion or rigid image fusion. In thecase of rigid image fusion, the relative position between the pixels ofa 2D image and/or voxels of a 3D image is fixed, while in the case ofelastic image fusion, the relative positions are allowed to change.

In this application, the term “image morphing” is also used as analternative to the term “elastic image fusion”, but with the samemeaning.

Elastic fusion transformations (for example, elastic image fusiontransformations) are for example designed to enable a seamlesstransition from one dataset (for example a first dataset such as forexample a first image) to another dataset (for example a second datasetsuch as for example a second image). The transformation is for exampledesigned such that one of the first and second datasets (images) isdeformed, for example in such a way that corresponding structures (forexample, corresponding image elements) are arranged at the same positionas in the other of the first and second images. The deformed(transformed) image which is transformed from one of the first andsecond images is for example as similar as possible to the other of thefirst and second images. Preferably, (numerical) optimisation algorithmsare applied in order to find the transformation which results in anoptimum degree of similarity. The degree of similarity is preferablymeasured by way of a measure of similarity (also referred to in thefollowing as a “similarity measure”). The parameters of the optimisationalgorithm are for example vectors of a deformation field. These vectorsare determined by the optimisation algorithm in such a way as to resultin an optimum degree of similarity. Thus, the optimum degree ofsimilarity represents a condition, for example a constraint, for theoptimisation algorithm. The bases of the vectors lie for example atvoxel positions of one of the first and second images which is to betransformed, and the tips of the vectors lie at the corresponding voxelpositions in the transformed image. A plurality of these vectors ispreferably provided, for instance more than twenty or a hundred or athousand or ten thousand, etc. Preferably, there are (other) constraintson the transformation (deformation), for example in order to avoidpathological deformations (for instance, all the voxels being shifted tothe same position by the transformation). These constraints include forexample the constraint that the transformation is regular, which forexample means that a Jacobian determinant calculated from a matrix ofthe deformation field (for example, the vector field) is larger thanzero, and also the constraint that the transformed (deformed) image isnot self-intersecting and for example that the transformed (deformed)image does not comprise faults and/or ruptures. The constraints includefor example the constraint that if a regular grid is transformedsimultaneously with the image and in a corresponding manner, the grid isnot allowed to interfold at any of its locations. The optimising problemis for example solved iteratively, for example by means of anoptimisation algorithm which is for example a first-order optimisationalgorithm, such as a gradient descent algorithm. Other examples ofoptimisation algorithms include optimisation algorithms which do not usederivations, such as the downhill simplex algorithm, or algorithms whichuse higher-order derivatives such as Newton-like algorithms. Theoptimisation algorithm preferably performs a local optimisation. Ifthere is a plurality of local optima, global algorithms such assimulated annealing or generic algorithms can be used. In the case oflinear optimisation problems, the simplex method can for instance beused.

In the steps of the optimisation algorithms, the voxels are for exampleshifted by a magnitude in a direction such that the degree of similarityis increased. This magnitude is preferably less than a predefined limit,for instance less than one tenth or one hundredth or one thousandth ofthe diameter of the image, and for example about equal to or less thanthe distance between neighbouring voxels. Large deformations can beimplemented, for example due to a high number of (iteration) steps.

The determined elastic fusion transformation can for example be used todetermine a degree of similarity (or similarity measure, see above)between the first and second datasets (first and second images). To thisend, the deviation between the elastic fusion transformation and anidentity transformation is determined. The degree of deviation can forinstance be calculated by determining the difference between thedeterminant of the elastic fusion transformation and the identitytransformation. The higher the deviation, the lower the similarity,hence the degree of deviation can be used to determine a measure ofsimilarity.

A measure of similarity can for example be determined on the basis of adetermined correlation between the first and second datasets.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, the invention is described with reference to theappended figures which give background explanations and representspecific embodiments of the invention. The scope of the invention ishowever not limited to the specific features disclosed in the context ofthe figures, wherein

FIG. 1 illustrates a flow diagram showing the basic steps of the methodaccording to the first aspect;

FIG. 2 illustrates deformation of an image representation of anatomicalstructures in the segmented patient image and boundary conditionstherefor; and

FIG. 3 is a schematic illustration of the system according to the sixthaspect.

DESCRIPTION OF EMBODIMENTS

FIG. 1 illustrates the basic steps of the method according to the firstaspect, in which step S11 encompasses acquiring the patient image data,step S12 encompasses acquiring the atlas data, step S13 encompassesacquiring the transformability model data, step S14 encompassesdetermining the segmented image data and subsequent step S15 encompassesdetermining the assigned transformability model data.

FIG. 2 illustrates deformation of a digital image representation (i.e. adepiction) of anatomical structures in the segmented patient image andboundary conditions therefor. A composite object may be defined to movemultiple objects (e.g. prostate, bladder and rectum), which is moved asa whole, i.e. a deformation field is created which overlays the union ofthe objects. When the grid is smoothly deformed, the objects which areconnected to the grid (i.e. the anatomical structures which have beenassigned to the geometric transformability model) are deformedsimultaneously.

However, not all objects are anatomically connected to each other. If anobject is moved, adjacent structures may either

-   -   be simultaneously moved, or    -   stay unchanged, or    -   restrict the movement of the object, or    -   move only when they are touched, or    -   slide along other objects, or    -   move in a way which is predetermined by anatomical restrictions        like sinew or joint.

One possible solution may be a bio-mechanical model which is constrainedby the anatomical variability of the human being. In such a model, notonly normal anatomically correct movements are possible, but alsomovements which correspond to anatomical variabilities, e.g. the heartcannot move in the thorax of an individual, but it can be at differentpositions for different individuals. Therefore, the method according tothe first aspect may allow moving the heart inside the thorax. However,it may not allow to separate the bladder from the prostate, since thisis not possible for any individual.

A model may be generated by statistically evaluating a set of patientimages. From the statistic variability, the motion/movability parametersand the model may be received. Evaluation of different scans of onepatient will lead to finding e.g. an arm in each patient image in adifferent position. From the resulting variability, a rotation of thearm may be determined. A computer program can do that fullyautomatically and on that basis calculate a model. That will then be amodel. If the same is done for scans from many different patients, onewill notice that the model does not only permit rotation of therespective arm but also a change of length of that arm because everyhuman being has a different arm length. Such a model will then have beengenerated in the same manner as the above-mentioned biomechanical modelbut will be based on a different set of images.

In a given segmentation of a patient image data set, e.g. a segmentedbladder, prostate, rectum and bone are present. These segmentationsappear as overlays over the gray value patient image data set so thatone can observe, whether there are differences between the segmentationresult and the position of image representations of anatomicalstructures in the patient image or not. If there are differences, theuser may want to correct these differences.

The method according to the first aspect includes the following approachfor allowing for such a correction:

-   -   A model of the human body is defined. This can be done e.g. by        defining a deformation grid, which overlays the organs but is        disconnected at vertices belonging to not connected organs as        shown in FIG. 2.    -   The grid is connected to the organs. When pulling or pushing the        grid (.e.g. manual user interaction with a pointer tool such as        a mouse or a touch screen for selecting e.g. a node of the grid        for changing its position, e.g. by drag-and-drop), the organs        also move accordingly. The grid has a certain elasticity        (defined e.g. by a spring model or finite element model), so        that the whole grid moves when pulling at some region. The        coefficients of this elasticity may not be driven by the real        elasticity of the tissue, but by a strength given by GUI input,        which defines a kind of range of the deformation. Distant        structures may be defined to move slower than nearby structures.        If one pulls at the top left corner of the bladder, the bladder        may be defined to move stronger than the prostate. And when        changing the position of the segmentation of the rectum, only        the part connected directly to the prostate moves slowly. The        other rectum parts move very slowly. The bone is not connected        at all and don't move. The structures may be defined to keep        their position after movement (e.g. when the user releases the        mouse button). The next movement may be defined to start with a        new regular grid. The movement action can then be repeated        several times.    -   The grid can be defined in the atlas and can be transferred to        the patient.    -   There are also border conditions, which have to be taken into        account, e.g. the prostate is not allowed to move into the bone.        Rather, it can only slide along the bone. There are in principle        two boundary conditions: 1) collision (no overlap of organs),        and 2) sliding with contact (two organs are always in contact,        but the contact point(s) can move, e.g. lung, liver and heart,        which move simultaneously, but along the ribs).    -   The distorted grid can also be used to distort the registration        (the atlas-patient mapping), which was the basis for the        segmentation: Reg_(new)=Distortion*Reg_(old). This registration        can be used afterwards to transfer other objects from the atlas        to the patient. If the newly transferred objects are in the        region of bladder, prostate or rectum, their segmentation can be        as well automatically corrected.

FIG. 3 is a schematic illustration of the medical system 1 according tothe sixth aspect. The system is in its entirety identified by referencesign 1 and comprises a computer 2, an electronic data storage device(such as a hard disc) 3 for storing at least the patient data and adisplay device 4 (such as a monitor). The components of the medicalsystem 1 have the functionalities and properties explained above withregard to the sixth aspect of this disclosure.

1.-15. (canceled)
 16. A computer-implemented method for determining thegeometric transformability of a segmentation of a medical patient image,comprising: acquiring patient image data which describes a patient imageof an anatomical body part of a patient; acquiring atlas data whichdescribes an image-based model of the anatomical body part and at leastone value of at least one parameter of a geometric transformabilitymodel of the anatomical body part; acquiring transformability model datawhich describes the geometric transformability model of the anatomicalbody part; determining segmented image data, based on the patient imagedata and the atlas data and the transformability model data, whichdescribes a segmentation of the patient image and a geometric relationbetween the geometric transformability model and the segmentation; anddetermining assigned transformability model data , based on thesegmented image data and the atlas data, which describes an assignmentof the at least one value of the at least one parameter to the geometricrelation between the geometric transformability model and thesegmentation.
 17. The method according to claim 16, wherein thegeometric transformability model describes a movement interactionbetween the relative positions of at least substantially disjunct partsof the image-based model of the anatomical body part.
 18. The methodaccording to claim 16, wherein the geometric transformability modeldescribes a movability for at least one of translational or rotationaldegrees of freedom, of the anatomical body part or an elastic orinelastic deformation model of the image-based model of the anatomicalbody part for a distance-dependent deformation function of theimage-based model of the anatomical body part.
 19. The method accordingto claim 16, wherein the geometric transformability model is defined ona numeric grid which is linked to at least substantially disjunct partsof the anatomical body part.
 20. The method according to claim 16,wherein the geometric transformability model comprises a shape modeldescribing allowed shapes and movement directions of at leastsubstantially disjunct parts of the image-based model of the anatomicalbody part.
 21. The method according to claim 16, wherein the assignmentof the at least one value of the at least one parameter to the geometricrelation between the geometric transformability model and thesegmentation is determined by inputting the at least one value of the atleast one parameter to the transformability model.
 22. The methodaccording to claim 16, wherein the segmented image data is determined bydetermining a positional transformation between the patient image of theanatomical body part and the image-based model of the anatomical bodypart.
 23. The method according to claim 22, wherein the positionaltransformation is determined by applying an image matching algorithm asan elastic or rigid image fusion algorithm, to the patient image dataand the atlas data.
 24. The method according to claim 16, whereinboundary conditions of the geometric transformability model are definedwithin and/or between at least substantially disjunct parts of theanatomical body part.
 25. The method of claim 16 further comprisingadapting a segmentation of the medical patient image; determining, basedon the segmented image data and the assigned transformability modeldata, changed segmented image data describing a changed segmentation ofthe patient image.
 26. A system for planning a medical procedure,comprising: at least one computer having at least one processor; whereinthe at least one processor is operable to: acquire patient image datawhich describes a patient image of an anatomical body part of a patient;acquire atlas data which describes an image-based model of theanatomical body part and at least one value of at least one parameter ofa geometric transformability model of the anatomical body part; acquiretransformability model data which describes the geometrictransformability model of the anatomical body part; determine segmentedimage data , based on the patient image data and the atlas data and thetransformability model data, which describes a segmentation of thepatient image and a geometric relation between the geometrictransformability model and the segmentation; and determine assignedtransformability model data, based on the segmented image data and theatlas data, which describes an assignment of the at least one value ofthe at least one parameter to the geometric relation between thegeometric transformability model and the segmentation; a display devicefor displaying at least one of the patient image data, the segmentedimage data or the changed segmented image data; wherein the at least onecomputer is operably coupled to at least one electronic data storagedevice for acquiring, from the at least one data storage device, atleast the patient image data, the atlas data and the transformabilitymodel data, and to the display device for issuing, to the displaydevice, a command for displaying at least one of the patient image data,the segmented image data or, the changed segmented image data.
 27. Anon-transitory computer readable medium comprising instructions storedthereon which when executed by at least one processor causes the atleast one processor to: acquire patient image data which describes apatient image of an anatomical body part of a patient; acquire atlasdata which describes an image-based model of the anatomical body partand at least one value of at least one parameter of a geometrictransformability model of the anatomical body part; acquiretransformability model data which describes the geometrictransformability model of the anatomical body part; determiningsegmented image data , based on the patient image data and the atlasdata and the transformability model data, which describes a segmentationof the patient image and a geometric relation between the geometrictransformability model and the segmentation; and determine assignedtransformability model data based on the segmented image data and theatlas data, which describes an assignment of the at least one value ofthe at least one parameter to the geometric relation between thegeometric transformability model and the segmentation.